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At  the  tactical  level.  Command  and  Control  (C2)  Agility  implies  that  command  staff  has  a  range 
of  well-developed,  and  understood,  concepts  of  operation  that  they  can  choose  from  for  their 
organizations.  The  development  of  alternative  operational  concepts,  even  to  adjust  an  operations 
centre  to  the  inclusion  of  new  or  changed  capabilities  can  be  problematic  at  the  unit  level.  For  a 
naval  platform,  changes  to  the  physical  environment  are  generally  both  difficult  and  expensive. 

In  order  to  support  naval  concept  development  and  experimentation.  Defence  R&D  Canada  has 
developed  the  concept  of  a  Maritime  Capability  Evaluation  Facility.  This  concept  would  provide 
the  reconfigurable  physical  infrastructure  to  conduct  operator  in  the  loop  experimentation  on  C2 
concepts,  thus  enabling  the  investigation  of  concepts  of  operation  early  in  the  naval  capability 
development  cycle.  A  prototype  facility  to  support  submarine  C2  research  has  been  developed 
and  used  to  investigate  a  wide  range  of  technologies  that  could  be  used  in  the  development  of  a 
full  Navy  capability.  This  paper  describes  the  prototype  and  a  number  of  the  technologies 
investigated. 
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1.0  INTRODUCTION 


Command  and  Control  (C2)  Agility  and  the  NATO  C2  Agility  Model  [1]  are  built  upon  the  concept  that 
C2  organizations  can  switch  organizational  structures  and  processes  to  best  match  the  C2  situation  facing 
them;  in  some  cases  using  multiple  C2  processes  simultaneously  for  different  collaborations  or  situations. 
At  the  tactical  level  time  scales  are  often  more  compressed  and  therefore  it  is  essential  that  command  staff 
have  well-developed  and  understood  options  available.  The  C2  Agility  Model  reinforces  the  importance 
of  concept  development  and  experimentation  that  were  heavily  promoted  over  the  past  decade  in  order  to 
provide  that  understanding.  However,  experimentation  at  the  tactical  level  can  be  problematic  due  to  the 
necessity  to  modify  equipment  or  platforms,  particularly  those  which  directly  interact  with  or  are  part  of 
weapon  systems.  On  naval  platforms  this  is  especially  true  since  even  small  changes  may  invalidate 
weapon  certification. 

As  navies  move  to  save  costs  by  crew  reduction,  the  concept  of  specialized  mission  crewing  options  and 
cross  training  of  crew  members  looks  attractive.  Concepts  such  as  these  that  are  investigated  in  isolation 
can  miss  the  complex  interactions  currently  in-place  between  primary  and  secondary  duties,  at-sea 
training,  quality  of  life  etc.  Recognition  that  the  tactical  C2  is  a  complex  system  is  important  and  leads  to 
a  requirement  for  holistic  experimentation  in  order  to  understand  likely  and  unintended  consequences  of 
changes  in  one  warfare  area  on  another  area  or  process  [2].  This  requirement  is  extant  whether  the  new 
concept  is  one  of  new  equipment,  tasks,  organization  or  critical  space  layout.  Offsetting  this  requirement 
for  in  depth  understanding  is  the  fact  that  technology  is  changing  much  faster  than  current  programs  can 
conduct  acquisition,  let  alone  concept  development  and  then  acquisition.  There  is,  therefore,  an  urgent 
need  for  processes  and  infrastructure  to  reduce  the  cost  and  timescale  of  experimentation  in  order  to  better 
tailor  acquisition,  or  process  change,  to  the  concepts  that  provide  realistic  and  practical  gains. 

Defence  R&D  Canada  (DRDC)  has  identified  a  requirement  for  the  capability  to  provide  evaluation 
support  to  the  ongoing  evolution  of  shipboard  command  and  control  [3].  This  evolution  can  come  in  a 
wide  variety  of  forms  ranging  from  changes  in  doctrine  or  personnel  to  major  changes  in  weaponry. 
DRDC  believes  that  this  requirement  is  applicable  across  the  Department  of  National  Defence  (DND). 

System  engineering  experience  [4]  indicates  that  early  full  system  evaluation  of  designs  is  required  in 
highly  complex  systems  since  sub-system  evaluation  may  not  be  predictive  of  the  overall  system  effect. 
It  has  been  shown  by  both  allies  and  industry  that  there  are  significant  savings  to  trialing  changes  to 
systems  before  implementation  in  operational  units.  However,  the  cost  of  full  system  evaluation  has  often 
been  deemed  prohibitive  for  all  but  the  largest  of  projects. 

While  capability  maintenance  packages  are  often  discussed  in  terms  of  new  technology,  it  is  well 
understood  that  often  the  actual  systems  are  a  complex  socio-technological  systems  of  systems  in  which 
the  commander  and  human  operators  are  critical  parts.  In  particular,  the  command  and  control  process 
supported  by  the  combat  control  system  (CCS)  is  such  a  system.  Thus,  any  changes  to  the  equipment, 
personnel,  training,  tactics,  procedures,  sensors,  algorithms  need  to  be  assessed  from  a  systems 
perspective. 

DRDC  has  developed  a  concept  called  the  Maritime  Capability  Evaluation  Laboratory  (MCEL)[3]  to 
provide  a  cost-effective  long-term  infrastructure  for  naval  C2  concept  evaluation.  The  basic  idea  is  to 
instantiate  an  infrastructure  model  that  minimizes  the  cost  of  entry  for  naval  concept  development  (or 
acquisition)  projects  to  conduct  evaluation  experimentation  as  part  of  the  systems  engineering  prior  to 
major  implementation  engineering  and  equipment  acquisition.  This  requires  the  construction  of  physical 
emulations  of  naval  critical  spaces  that  are  readily  available  for  use  and  have  associated  baseline 
warfighting  performance  measures.  Despite  advances  in  virtual  worlds,  C2  remains  an  inherently  human 


exercise  that  requires  a  level  of  human  interaction  that  is  not  currently  available  except  by  putting  the 
team  into  an  emulation  of  its  physical  environment. 

The  laboratory  is  expected  to  support  the  full-scale  implementation  of  the  ship’s  C2  systems  and  critical 
spaces;  while  utilizing  modelling  and  simulation  to  implement  sensors  and  weapon  effects.  The  level  of 
environmental  fidelity  will  depend  upon  the  requirements  of  the  capability  being  tested,  but  it  is  expected 
that  MCEL  will  make  use  of  Royal  Canadian  Navy  (RCN)  standard  models  whenever  possible.  System 
components  fidelity  will  also  depend  upon  requirements.  The  laboratory  must  also  include  control, 
monitoring  and  data  collection  instrumentation  to  evaluate  the  effectiveness  of  human  operators,  and 
networking  capabilities  to  allow  it  to  be  linked  with  other  simulation  and  training  facilities,  including 
ships  alongside.  Integral  to  this  concept  is  the  ability  to  baseline  current  capability  so  that  the  effects  of 
new  concepts  and  processes  can  be  fully  evaluated. 

DRDC’s  mandate  within  DND  is  to  investigate  new  concepts  and  technology.  Thus,  in  2008,  DRDC 
initiated  a  project  with  two  objectives,  firstly  to  investigate  new  operations  room  concepts  for  the 
VICTORIA  Class  submarine  [5,6]  and  secondly  to  develop  a  prototype  MCEL  infrastructure  within 
which  to  conduct  performance  experimentation  on  developed  concepts  and  investigate  enabling 
technologies.  This  paper  addresses  some  of  the  lessons  learned  from  the  VICTORIA  Capability 
Evaluation  Laboratory  (VCEL)  which  resulted  from  the  second  objective. 

2.0  VICTORIA  CAPABILITY  EVALUATION  LABORATORY  (VCEL) 

The  VICTORIA  Capability  Evaluation  Laboratory  (VCEL)  is  a  prototype  command  and  control  concept 
evaluation  experimentation  laboratory.  Originally  named  the  virtual  VICTORIA  or  vVICTORIA  it  has 
evolved  from  work  conducted  by  allies  such  as  the  Australian  virtual  Collins. 

The  general  concept  is  to  provide  a  flexible  and  adaptable  emulation  of  the  VICTORIA  class  submarine 
operations  centre  within  which  investigators  can  cost  effectively  modify  the  environment  to  expose 
operational  personnel  to  new  command  and  control  concepts.  The  level  of  experimentation  to  be 
conducted  in  the  VCEL  is  at  the  point  where  a  full  war-fighting  team  is  required.  That  is  to  say  that  while 
individual  usability  assessments  could  use  VCEL,  it  is  really  aimed  at  enabling  the  evaluation  of  concepts 
that  impact  a  significant  portion  the  whole  command  or  operating  centre  team. 

This  human-in-the-loop  level  of  experimentation  imposes  a  number  of  constraints: 

1.  the  physical  limitations  of  the  space  must  be  realistically  represented; 

2.  all  inter-personal  communications  modalities  need  to  be  represented; 

3.  all  expected  sensor/information  stimuli  must  have  realistic  levels  of  availability;  and, 

4.  all  individual  and  team  behaviours,  communications  and  interactions  must  be  recordable. 

Erom  the  start  of  the  project  the  design  concept  was  for  a  full  scale  physical  mock-up  of  the  VICTORIA 
class  submarine  operations  centre.  This  is  in  contrast  to  many  training  systems  which  can  incorporate 
30%  or  more  extra  space  to  accommodate  trainers  and  observers.  In  the  case  of  VCEL  it  was  felt  that  an 
important  part  of  operational  procedures  were  the  constraints  imposed  by  the  physical  space.  The 
physical  mock-up  would  be  populated  with  enough  of  the  physical  equipment  to  enable  the 
experimentation  requirements  of  the  concept  evaluations  being  developed  by  the  concept  development 
team.  A  system  design  that  allowed  a  variable  fidelity  in  equipment  emulation,  alongside 
experimentation  data  collection,  was  therefore  required.  Eigure  1  provides  a  concept  diagram  for  the 
implemented  software  system.  A  key  part  of  the  final  system  was  the  use  of  a  version  of  the  commercial 


naval  game  Dangerous  Waters  produced  by  Sonalysts[7]  to  provide  background  computer  generated 
forces,  scenario  implementation,  and  initial  operating  capability  for  internal  systems. 


Figure  1:  VCEL  software  system  diagram. 

At  the  centre  of  the  system  is  free  and  open  source  server  for  implementing  messaging  between  software 
components,  ActiveMQ[8].  Using  this  technology  allowed  the  integration  of  a  wide  variety  of  systems, 
ranging  from  actual  boat  systems  to  in-house  developed  emulated  systems. 

At  the  top  of  the  diagram  are  the  system  management,  monitoring,  and  logging  applications.  In  VCEL, 
these  were  all  written  in-house  to  meet  our  specific  needs.  There  is  also  a  box  for  specialized  simulations 
and  data  providers;  two  examples  in  VCEL  are  a  data  provider  that  reads  sound-speed  profiles  and  responds 
to  queries,  and  a  very  basic  Automatic  Information  System  (AIS)  network  simulation  that  creates  AIS 
reports  for  other  software  to  consume. 


Figure  2:  The  ECPINS  system  [OSI]. 


To  the  right  are  real  systems  that  are  integrated  with  VCEL.  Currently  we  only  support  the  ECPINS-W 
commercial  off-the-shelf  (COTS)  electronic  charting  and  navigation  system.  In  principle,  anything  could  be 
connected,  though  an  adaptor  would  be  required  to  interface  with  ActiveMQ.  Figure  2  shows  the  ECPINS 
station  in  VCEL. 

At  the  bottom  of  Figure  1  is  a  box  representing  experimental  software.  In  the  VCEL  context,  this  means 
prototype  interfaces  or  entire  systems  meant  for  operators  to  use  directly.  We  have  been  very  successful 
using  Adobe  Flash  and  Apache  Flex  as  rapid  application  development  (RAD)  tools  for  creating  new 
operator  interfaces  and  systems,  and  for  emulating  real  systems  where  it  is  impractical  to  use  them  directly. 
The  Flash  based  operator  interfaces  were  constructed  as  web  applications  and  connected  to  ActiveMQ  via 
an  open  source  remoting  and  messaging  technology  known  as  BlazeDS[9].  The  only  architectural  exception 
was  for  mapping  in  which  the  open  source  MapServer[10]  platform  was  used  to  drive  map  based  display 
elements  directly.  Figure  3  shows  the  architecture  for  activating  the  Flash  interfaces  from  the  simulation. 


Figure  3:  Interface  Activation  Architecture. 

Finally,  on  the  left  of  Figure  1  is  a  modified  version  of  Sonalysts’  “Dangerous  Waters[l]”  modem  naval 
combat  game.  This  software  provides  all  operator  interfaces,  periscope,  sonar,  radar,  electronic 
countermeasures  (ECM),  weapons  and  effects,  ship  motion  simulation,  as  well  as  an  extensive  scenario 
editor.  While  it  does  not  directly  simulate  a  VICTORIA  class  submarine,  it  does  provide  representative 
stations  where  real  naval  operators  can  do  their  jobs. 

Communications  systems  use  their  own  client-server  software,  and  as  do  some  of  the  independent  data 
collection  systems. 

Following  the  system  design,  all  experimental  control,  data  collection  and  observation  is  conducted 
outside  the  shell  of  the  physical  space  emulation.  Thus,  the  experiment  participants  are  isolated  from 
interference  from  observers  and  other  exercise  control  activities. 


3.0  VCEL  TECHNOLOGIES 


During  the  implementation  of  the  VCEL  the  team  investigated  a  wide  range  of  technologies  and  gathered 
ideas  from  collaborators  in  several  allied  nations.  For  example,  the  team  had  several  opportunities  to 
discuss  ideas  with  the  technical  team  at  DSTO’s  Maritime  Experimentation  Laboratory  (MEL)  and 
ANZAC  Combat  System  Integration  Laboratory  (ACSIL).  Much  of  the  advice  and  ideas  gleaned  were 
anecdotal  and  do  not  appear  to  have  been  reported  in  the  open  literature.  In  the  spirit  of  that  exchange,  we 
do  not  claim  any  special  insight,  but  rather  wish  to  pass  on  to  others  some  of  our  technical  experiences. 

3.1  Theatre  Set  Design  Technologies 

Early  in  the  project  definition  process  the  VCEL  team  recognized  that  there  was  a  great  deal  of  similarity 
between  theatrical  and  television  set  usage  and  the  proposed  usage  of  VCEL.  Thus,  one  of  the  main  areas 
of  expected  technology  leverage  was  from  the  entertainment  industry. 

VCEL  Shell  Emulation 


In  many  cases  operations  centre  mock-ups  are  developed  within  facilities  that  have  been  designed  as 
‘‘computer”  facilities.  In  general  these  facilities  have  been  built  (whether  security  shielded  or  not)  with 
standard  8  foot  ceilings  and  minimally  raised  floors.  Portable  dividers  are  sometimes  used  to  mimic 
bulkheads  when  the  facility  is  much  larger  than  the  required  space.  A  number  of  issues  with  this  type  of 
facility  have  been  reported  anecdotally. 

i.  while  the  idea  of  the  raised  floors  is  to  keep  network  wiring  and  electrical  power  distribution 
out  of  the  way,  often  the  floors  are  only  minimally  raised  in  order  to  retain  as  much  headroom  as  possible. 
This  has  been  found  to  impede  the  ease  of  rewiring  as  multiple  floor  panels  must  be  removed  each  time. 

ii.  the  removal  of  floor  panels  to  access  wiring  and  centre  floor  power  distribution  tends  to  be 
problematic  as  there  is  almost  always  a  console  or  other  piece  of  furniture  that  must  be  moved  in  order  for 
the  floor  panel  to  be  removed. 

iii.  when  permanent  walls  exist,  the  peripheral  power  and  networking  is  often  permanently 
installed  in  them  which  then  causes  problems  with  maintenance  as  consoles  have  to  be  moved  in  order  to 
access  the  walls.  Note:  this  is  an  actual  issue  for  maintenance  in  many  operations  centres  as  well,  but  is 
compounded  in  a  facility  that  is  meant  to  be  flexible  and  adaptable. 

Following  discussions  with  colleagues  at  DSTO,  the  team  contracted  a  television  set  designer  to  assist  in 
developing  a  modular  and  flexible  design  which  was  then  implemented  under  a  second  contract  [11].  The 
resulting  design  (See  Figure  4)  incorporates  a  number  of  features  to  address  issues  mentioned  above. 

a.  All  around  access  to  the  operations  room  mock-up’s  shell.  By  using  the  television  stage  concept  of 
building  the  set  within  a  larger  space  it  is  possible  to  provide  access  from  all  directions. 

i.  Using  theatrical  stage  design  the  set  is  raised  to  a  level  where  it  is  feasible  to  access  the  floor 
from  outside  (not  inside).  In  VCEL’s  case  the  floor  is  raised  18  in  (25cm)  off  the  floor  which 
gives  sufficient  room  for  technicians  to  use  a  mechanics  crawler  to  move  about  underneath. 

ii.  The  floor  is  supported  on  Sin  (7.5cm)  diameter  aluminum  pipes  set  into  a  2x8  frame.  In 
terms  of  design  the  pipes  could  easily  be  replaced  with  longer  or  shorter  lengths  to  fit  a  space. 

iii.  The  walls  all  have  removable  panels  to  enable  access  to  the  backs  of  consoles. 


iv.  The  ceiling  has  cut-outs  in  the  framing  modules  to  allow  ease  of  wiring. 

V.  The  overall  ceiling  of  the  structure  was  re-enforced  to  support  personnel  and  has  access 
panels. 

b.  Major  construction  is  with  %  inch  (75mm)  good-both-sides  plywood.  This  provides  an  easily 
workable  and  replaceable  material  -  if  a  hole  is  required  it  is  easy  to  make  and/or  repair. 

i.  The  floor  is  a  double  layer  of  plywood  on  top  of  the  2x8  timber  framework,  providing  a  very 
stable  and  quiet  platform. 

ii.  Wall  frames  were  placed  to  emulate  approximate  location  of  hull  framing  with  curved 
sections  cut  from  %  in  plywood.  Exterior  walls  and  removable  panels  were  also  of  %  inch 
plywood. 

iii.  Interior  walls  are  %  in  plywood,  which  while  thicker  than  the  metal  walls  of  a  naval  platform 
are  much  closer  in  dimensions  than  residential  construction  methods. 

c.  Use  of  foam  panels  or  thin  plywood  for  curved  and/or  complicated  surfaces.  In  the  case  of  VCEL  Va 
plywood  was  used  to  emulate  the  curved  surface  of  the  pressure  hull.  Using  metal  guide  channels 
(standard  home  siding  accessories)  the  plywood  can  easily  be  slid  in  or  out  through  the  openings  on 
the  sides  of  the  facility.  (See  Eigure  4) 


Figure  4:  The  VCEL  experimental  space. 

Overall  the  design  was  both  easy  to  construct  in  modules  and  went  together  extremely  quickly  [4].  It 
remains  to  be  determined  how  easily  it  will  come  apart.  The  VCEL  shell  was  not  constructed  with  the 
intention  of  frequent  changes  to  wall  positions.  Some  experimentation  will  be  required  to  determine  how 
the  basic  set/stage  technology  would  best  be  used  if  multiple  ship  classes  or  ship  compartments  were  to  be 
supported  from  the  same  set/stage.  Certainly  the  floor  structure  would  provide  a  good  stable  foundation 
upon  which  modular  walls  could  be  installed. 

Another  issue  of  importance  is  the  integration  of  the  shell  with  the  building  infrastructure.  The  VCEL 
differs  from  a  television  set  in  its  life  span.  Many  television  sets  have  a  limited  lifespan,  while  navies 
tend  to  keep  ships  for  decades,  thus  there  were  a  number  of  elements  of  the  original  design  that  had  to  be 
adapted  to  longer  term  usage  and  government  fire  regulations: 


1.  While  the  design  called  for  the  plywood  to  be  sealed  with  paint  there  was  a  concern  about  the  overall 
amount  of  combustible  material  and  a  fire-retardant  paint  was  required  for  all  surfaces  (inside, 
outside,  underneath  etc.), 

2.  The  original  concept  was  for  all  power  and  network  distribution  to  come  from  outside  (i.e.,  extension 
cords).  However,  the  fire  marshal  mandated  that  power  be  provided  permanently  within  the  structure. 
In  a  future  instantiation  with  movable  walls  this  would  necessitate  installation  of  power  within  the 
raised  floor  structure. 

3.  In  usage  of  the  facility  the  team  has  also  determined  that  more  permanent  networking  within  the  shell 
structure  connected  to  a  patch  panel  would  simplify  maintenance  and  adherence  to  security  protocols. 

4.  While  the  overall  building  that  contains  VCEL  has  environmental  controls  it  was  recognized  early  on 
that  with  a  large  number  of  personnel  and  electrical  equipment  in  an  enclosed  space  that  the  shell 
would  need  environmental  control  as  well.  Thus,  an  early  addition  to  the  interior  was  the  inclusion  of 
air  ducting  to  match  that  of  the  real  VICTORIA  class  boats  which  was  tied  into  the  building’s 
systems. 

5.  Another  issue  that  came  up  a  number  of  times  was  the  need  for  an  internal  grid  system  on  the  floor  to 
assist  in  the  placement  of  consoles  and  other  equipment  and  the  calibration  of  video  data  collection 
systems.  Essentially  the  team  had  to  retroactively  mark  out  a  grid  on  the  floor  of  the  shell  after 
construction  which  would  have  been  much  easier  to  have  done  during  construction;  marked  and  then 
sealed  with  a  clear  finish  so  the  marks  are  not  worn  off  with  use. 

Console  Emulation 


As  with  the  Shell,  theatre  construction  techniques  were  used  to  create  mock-ups  of  the  major  console 
components  within  the  operations  centre  [12].  A  combination  of  plywood,  plastic  sheeting  and  fibreglass 
was  used  to  create  consoles  that  dimensionally  match  actual  equipment.  All  consoles  were  mounted  on 
lockable  wheels  so  that  they  can  easily  be  rearranged.  The  downside  of  this  mobility  is  the  need  of  the 
floor  grid  positioning  system  mentioned  above. 

The  construction  philosophy  was  to  build  the  general  physical  form  of  consoles  out  of  good  two  side 
plywood  (finished  to  the  correct  colour  scheme)  leaving  the  console  display  surfaces  open.  The  display 
surfaces  were  then  covered  with  Va  inch  (6.5  mm)  plastic  panels.  The  plastic  panels  were  easier  to  mount 
and  remove  than  plywood  while  retaining  the  ease  of  shaping  to  fit  to  control  features  and  displays  (see 
Eigures  2  and  5  for  examples). 

In  line  with  the  general  project  philosophy  to  only  model  to  the  level  required  for  the  planned 
experimentation,  only  console  controls  expected  to  be  required  were  instantiated.  This  was  motivated 
both  by  the  amount/complexity  of  controls  in  the  operations  room  and  the  mandate  to  explore  ‘cost 
effective”  technologies.  Early  on  in  the  project  the  console  construction  team  investigated  obtaining  the 
actual  multi-function  switches  used  in  one  of  the  consoles  with  the  intention  of  connecting  them  to  a 
simple  controller  that  could  be  activated  by  software  later.  The  cost  of  the  switches  alone  was  more  than 
the  cost  of  the  entire  shell.  Eollowing  this  discovery  it  was  decided  to  be  extremely  careful  in  picking 
which  switches  to  emulate,  and  wherever  possible  to  emulate  them  in  software  on  touch  panels. 

An  exception  to  this  policy  was  some  of  the  rotary  control  switches  on  one  of  the  standard  control 
consoles;  there  was  a  desire  to  provide  the  actual  physical  control  mechanism  but  the  actual  switches  were 
prohibitively  expensive,  the  solution  came  from  a  similar  control  device  used  for  arcade  games  which 


were  obtained  at  a  fraction  of  the  cost.  The  use  of  gaming  technology  in  general  is  discussed  in  more 
detail  below. 

In  the  cases  where  the  controls  or  displays  were  determined  not  to  be  required  as  stimulus  for  the  project’s 
experiment  it  was  decided  to  use  full  scale  pictures  of  the  controls  attached  to  the  control  surfaces  instead 
of  implementing  them.  This  preserves  the  appearance  and  environment. 

An  issue  that  has  arisen  with  the  mobile  plywood  construction  is  that  in  some  cases  the  consoles  have  had 
stability  issues  due  to  heavy  displays  in  the  upper  parts  without  the  corresponding  heavy  hardware  in  the 
lower  casing.  When  coupled  with  the  consoles  being  on  wheels  rather  than  firmly  attached  to  the  floor  or 
bulkhead,  this  has  required  some  mobility  compromises. 

3.2  Centralized  Simulation/Emulation  technologies 

Based  on  the  VCEL  team’s  experience  with  distributed  simulation  experiments,  one  of  the  technology 
areas  of  particular  interest  was  that  of  simplifying  experimentation  setup  and  system  configuration. 

Simply  put,  human-in-the-loop  experimentation  with  teams  generally  requires  a  large  number  of 
relatively  independent  systems  that  must  be  networked  together.  This  implies  the  requirement  to 
coordinate  the  configuration  of  all  those  systems.  For  example,  the  start-up  of  the  VCELsystem  requires 
accessing  a  large  number  of  tools,  services,  applications  and  other  software  on  a  variety  of  physical  and 
virtual  computers,  often  in  a  specific  order  which  can  take  a  significant  amount  of  time.  Automation  of 
these  processes  to  save  time  and  reduce  the  number  of  human  errors  was  of  high  importance. 

The  initial  technology  investigated  was  the  use  of  server  based  virtual  machines  and  thin  clients.  The 
concept  was  for  all  console  displays  and  controls  to  be  on  thin-clients  driven  from  a  central  server  thus 
reducing  (or  eliminating)  the  requirement  for  significant  computer  hardware  in  consoles  within  the  shell. 
Further,  if  the  drivers  were  run  within  virtual  machines  on  the  server  than  the  configuration  of  the  system 
might  be  saved  as  a  set.  While  this  does  not  markedly  reduce  the  initial  setup  and  configuration,  bringing 
a  previous  experimental  setup  back  up  on  the  system  could  be  much  easier.  Unfortunately,  as  the  system 
design  developed  a  number  of  issues  arose: 

1.  elements  of  the  system  required  access  to  graphics  hardware  that  the  virtual  machines  would 
not  emulate; 

2.  the  lag  on  user  interface  interactions  was  too  long; 

3.  there  was  difficulty  running  multiple  applications  on  the  same  thin  client; 

4.  the  thin  clients  turned  out  be  more  fragile  than  expected;  and, 

5.  the  move  to  touch  screens  to  emulate  controls  meant  that  the  number  of  displays  where  the 
thin  clients  could  be  used  was  minimal. 

In  the  final  system  design  virtual  machines  are  used  extensively  to  host  appropriate  elements  but  local 
computers  were  required  for  all  consoles.  However,  an  alternate  open  source  scripting  technology 
(StAFF/STAX[13])  has  been  used  successfully  to  reduce  the  start-up/shut-down  time.  In  addition,  the 
use  of  a  common  simulation  system.  Dangerous  Waters,  reduced  the  overall  configuration  problem. 

3.3  Gaming  Technologies 

The  VCEL  team  has  been  investigating  the  use  of  commercial  gaming  technology  and  serious  games  for 


experimentation  for  a  number  of  years  [14].  In  our  experience  the  technology  has  a  number  of  attributes 
that  make  it  useful: 


1.  First  and  most  obvious  is  the  low  cost.  Compared  to  traditional  simulation  systems,  serious 
game  technology  is  extremely  economical  (by  orders  of  magnitude  in  many  cases).  Even  in 
cases  where  a  commercial  game  cannot  be  used  off-the-shelf  but  requires  modification,  the 
cost  is  still  much  lower  than  that  of  traditional  simulation  systems; 

2.  Another  attribute  of  game  technology  is  how  accessible  it  is  to  end-users.  Not  only  do 
trainees  (or  in  our  case  experiment  participants)  find  it  easy  to  pick  up  and  use,  so  do  scenario 
developers,  instructors,  and  system  administrators.  This  accessibility  comes  from  the  simple 
fact  that  games  must  be  easy  to  use  out-of-the-box,  without  instruction,  or  else  they  have  no 
market;  and, 

3.  A  third  attraction  of  games  is  that  we  can  often  find  a  game  that  does  most  or  all  of  what  we 
require  from  a  simulation  system. 

Of  course,  game  technology  does  have  some  drawbacks.  The  fidelity  of  the  simulation  itself  is  often  not 
to  the  standard  of  traditional  simulators  as  the  need  to  run  on  consumer  hardware  is  of  paramount 
importance,  and  gameplay  is  more  important  than  fidelity.  Game  technology  is  often  built  on  a  closed 
architecture,  and  this  can  make  it  very  difficult  to  take  a  commercial  off-the-shelf  (COTS)  game  and 
interface  with  existing  simulation  systems.  For  example,  in  the  training  or  simulation  context  it  is  easy  to 
imagine  a  need  to  interface  with  a  Distributed  Interactive  Simulation  (DIS)  or  High  Level  Architecture 
(HLA)  simulation,  and  there  are  very  few  COTS  games  that  can  do  this  out  of  the  box. 

Dangerous  Waters  by  Sonalysts 

While  the  original  simulation  system  concept  was  an  HLA  based  distributed  simulation  using  a  set  of  in- 
house  simulation  federates,  licenses  for  a  naval  serious  game  (Dangerous  Waters  by  Sonolysts)  became 
available  early  in  the  project  timeline.  The  Dangerous  Waters  game  had  been  investigated  by  the  team 
previously  and  another  DRDC  project  had  worked  with  Sonolysts  to  develop  a  version,  known  as  DW- 
MSEAS,  with  a  several  significant  enhancements  compared  to  the  commercial  version  (including  HLA 
compatibility). 

Dangerous  Waters  (DW)  is  a  commercial  game  that  was  first  released  in  2006  and  is  still  available  at  a 
cost  of  approximately  $15  per  seat.  There  are  many  features  of  DW  that  make  it  particularly  attractive  in 
the  maritime  context: 

1.  First,  it  includes  simulations  of  many  platforms:  maritime  patrol  aircraft,  maritime 
helicopters,  surface  ships,  and  submarines; 

2.  It  includes  a  comprehensive  scenario  generation  tool; 

3.  While  not  providing  all  of  the  consoles  or  capabilities  of  any  naval  platform,  DW  does 
provide  each  type  of  console.  Thus,  while  DW  provides  only  a  single  sonar  console,  that 
console  has  multiple  screens  that  provide  access  to  the  breadth  of  submarine  sonar 
capabilities  at  a  reasonable  level  of  fidelity; 

4.  Additionally,  DW  provides  a  unique  multiplayer  mode  where  each  crew  station  (Figure  5)  on 
the  controllable  platforms  can  be  run  by  a  separate  player.  This  was  critical  to  our 


experimentation  because  we  needed  to  have  a  complete  command  crew  doing  their  jobs 
within  the  control  centre  of  the  submarine;  and, 

5.  DW  comes  with  an  after  action  replay  system  that  records  reconstruction  level  data. 

While  the  HLA  interoperability  with  the  Real  Time  Reference  (RPR)  Federation  Object  Model  (FOM) 
solved  some  of  the  design  issues  there  were  other  requirements  that  required  access  to  internal  model  data 
that  are  not  covered  by  the  RPR  FOM  and  thus  Sonalysts  was  contracted  to  create  an  interface  that  allows 
access  to  a  wider  variety  of  internal  simulation  state  data. 

DW  essentially  provided  all  of  the  core  functionality  required  to  simulate  the  operation  of  a  submarine. 
DW  includes  an  overall  tactical  picture,  electronic  warfare,  radar,  radios,  ship  control,  periscope,  narrow 
and  broadband  passive  sonar,  active  sonar,  as  well  as  target  motion  analysis  and  fire  control.  When  the 
submarine  is  operating  on  the  surface,  there  is  also  a  sail-bridge  position  available.  Under-the-hood  DW 
includes  an  underwater  acoustics  model,  environmental  modelling  of  currents,  time-of-day,  sea  state, 
wind,  rain  and  cloud  levels,  and  everything  else  needed  for  a  simulation  of  the  physical  environment. 

From  a  content  perspective,  DW  also  provided  us  with  enough  to  get  started.  While  the  VICTORIA  class 
submarine  is  not  a  controllable  platform  within  DW,  there  are  several  controllable  submarine  platforms 
that  can  stand  in  for  the  VICTORIA  class.  In  our  current  experiment,  we  are  making  use  of  the  LOS 
ANGELES  class  controllable  submarine  within  DW  to  simulate  the  VICTORIA  class  workstations.  DW 
also  comes  with  a  complete  terrain  database  for  the  world  between  85°  north  and  75°  south  latitude  (this 
latitude  range  is  larger  than  is  typical  of  many  COTS  games). 


Figure  4:  The  target  motion  analysis  and  fire  control  stations  inside  VCEL 

As  the  system  came  together  and  experimentation  support  activities  were  begun,  other  benefits  to  using 
DW  have  come  to  light.  In  particular. 


1.  The  scenario  generation  system  is  quite  easy  to  use  particularly  compared  that  in  a  number  of 
legacy  training  systems; 

2.  The  DW  periscope  simulation  was  determined  to  provide  as  good  performance  as  the  majority  of 
third  party  or  standalone  simulators  [15];  and, 

3.  Sonalysts  have  been  very  responsive  in  providing  extensions  to  the  interface  to  provide  access  to 
additional  internal  state  variables. 

It  should  be  pointed  out  that  DW  has  not  been  a  panacea  as,  for  example,  it  does  not  have  a  semi- 
automated  forces  capability.  Platforms  are  either  controllable,  needing  human  control,  or  scripted. 
However,  it  has  provided  a  great  deal  of  capability  out-of-the-box  which  enabled  the  team  to  move  onto 
other  issues  that  otherwise  would  not  have  gotten  addressed. 

3.4  Consumer  hardware  used  in  VCEL 

All  of  the  computer  equipment  used  in  VCEL  is  COTS,  and  most  of  that  is  consumer  class  hardware  that 
can  be  purchased  at  any  computer  retailer.  Exceptions  to  this  are  the  network  switches,  which  are 
enterprise-grade,  and  the  rack-mount  equipment  which,  while  still  COTS,  is  lower-end  enterprise  grade. 

In  the  past  we  have  found  that  the  types  of  network  switches  sold  for  household  use  often  cannot  handle 
the  kind  of  throughput  required  by  simulation  systems. 

The  team  also  tried  ultra-low-cost  mini-systems  for  inside  the  consoles.  In  spite  of  pre-testing, 
component  quality  is  an  issue.  The  team  has  found  that  the  savings  in  initial  cost  is  not  worth  the 
maintenance  issues  that  arise  from  lower  quality  hardware.  Eor  example,  it  can  take  a  long  time  and 
effort  to  track  down  issues  with  intermittent  faults  in  things  like  network  cards.  While  these  issues  are  not 
generally  a  problem  in  a  home  system,  complex  distributed  simulations  can  be  quite  sensitive  to  network 
and  hardware  issues. 

3.5  Data  Collection  Systems 

Since  the  purpose  of  the  VCEL  is  to  conduct  experimentation,  data  collection  systems  are  a  major 
element  of  the  overall  system.  In  many  experimentation  and  training  facilities  extra  space  is  built  into  the 
facility  to  allow  for  observation  of  participant  behaviour.  However,  from  the  outset  the  intent  of  VCEL 
was  to  enable  the  inclusion  of  ergonomic  issue  experimentation  which  meant  that  the  mock-up  could  have 
no  extra  space.  Thus,  the  data  collection  system  had  to  enable  remote  observation  and  data  collection 
from  outside  the  shell,  and  do  so  in  the  low-light  conditions  often  found  in  an  operations  centre. 

The  initial  solution  was  the  inclusion  of  multiple  low-light  surveillance  quality  camera  systems  with  both 
recording  and  display  on  large  screen  displays  for  observers.  In  addition,  an  open  source  screen  capture 
system,  Taksi  [14]  has  been  attached  to  all  consoles.  The  downside  of  Taksi  is  that  it  is  not  start-up 
automatable.  These  systems  provide  generalized  observation  and  recording  functionality. 

On  the  audio  side  the  system  uses  a  radio  system  simulator  developed  by  the  Canadian  Army  (SimSpeak) 
to  provide  the  internal  communications  network.  SimSpeak  uses  its  own  client-server  software  and 
includes  a  recording  function.  All  participants  are  fitted  with  individual  MP3  format  recording  devices  to 
provide  individual  non-communications  audio  records.  The  MP3  players  are  cheap,  durable,  and 
eliminate  the  security  issues  with  wireless  devices.  Eigure  6  shows  one  of  the  commercial  MP3  players 
used  in  VCEL. 


Figure  5:  Commercial  MP3  player  used  for  voice  recording. 

These  systems  provide  good  coverage  of  console  operators  and  communications.  However,  as  the 
concept  development  team  developed  their  experimentation  requirements  it  was  realized  that  a  system 
was  required  to  track  the  location  of  the  watch  officer,  where  they  were  looking,  and  if  looking  at  a 
particular  display,  what  part  of  the  display.  The  team  had  available  stationary  eye  tracking  systems  that 
had  been  used  for  previous  experiments  but  the  combination  of  a  non-stationary  participant  and  no 
wireless  led  to  a  need  to  look  at  mobile  eye  tracking  solutions.  Following  a  competitive  bid  process  a  set 
of  SMi  eye  tracking  glasses  were  obtained  to  provide  the  in-display  tracking.  This  system  consists  of  a 
pair  of  light  weight  glasses  with  an  eye  tracking  sensor,  field  of  view  camera  and  audio  recording.  All 
data  are  recorded  on  a  portable  recording  unit  that  can  be  attached  to  the  participant’s  belt.  Up  to  three 
hours  of  data  can  be  recorded  in  a  session.  Unfortunately  the  system  does  not  allow  for  real-time 
monitoring  of  the  data. 

In  order  to  digitally  track  participant  location  and  movement  the  team  turned  again  to  gaming  technology; 
in  this  case  the  Microsoft  Kinect  system.  Using  in-house  software  developed  for  VCEL  and  a  set  of  four 
Kinect  sensors,  the  system  can  automatically  track  the  position,  posture  and  orientation  of  the  mobile 
participants.  Figure  6  is  a  screen  capture  of  the  Kinect  data  as  it  is  collected  in  real-time. 


Figure  6:  Screen-shot  of  the  Kinect  based  data  acquisition  system  in  action. 


3.6  Rapid  Prototyping 


While  Dangerous  Waters  provided  the  initial  operating  capability  for  console  displays  in  VCEL,  the 
design  was  always  to  incrementally  replace  the  game  emulations  with  either  actual  software  or  in-house 
emulations  to  instantiate  all  the  consoles  in  the  operations  centre.  In  addition,  it  was  recognized  that 
many  concepts  of  interest  would  require  new  screen  concepts  and  designs.  Thus,  the  team  has 
investigated  a  couple  of  rapid  prototyping  systems. 

The  system  requirements  included  not  only  the  ability  to  rapidly  develop  new  screen  designs  but  to  be 
able  to  activate  them  from  the  underlying  simulation  system. 

The  team  has  found  that  the  combination  of  Adobe  Flash  with  BlazeDS  for  client-server  communication 
is  quite  effective.  Initially  there  was  some  hope  that  the  abundance  of  online  Flash  graphics  libraries 
would  provide  an  efficiency  advantage.  However,  military  systems  have  been  found  to  use  a  lot  of 
custom  interfaces  and  there  were  fewer  pre-built  gauges,  widgets,  and  other  UI  components  than  initially 
hoped.  This  was  also  compounded  by  the  fact  the  UI  components  had  to  exactly  match  the  concept 
designs.  Indeed,  legacy  military  interfaces  are  often  quite  compact,  unique  and  combine  a  large  amount 
of  functionality  from  a  small  number  of  controls.  This  has  resulted  in  the  activation  of  some  displays 
(especially  the  touch  screen  replacements  for  physical  switches)  being  more  complex  than  originally 
expected. 

On  the  other  hand  the  mapping  client  in  Flash,  OpenScales,  was  found  to  be  quite  useful,  and,  once  the 
hurdle  of  understanding  geographic  data  formats  was  overcome,  integration  of  maps  into  tactical  displays 
has  gone  well.  MapServer  makes  extensive  use  of  lengthy  configuration  files  (.MAP)  to  explicitly  define 
all  aspects  of  the  map  including  chart  data  sources,  size/extent,  layers,  points,  boundaries  and  how  each 
object  is  to  be  rendered.  Generating,  editing,  and  troubleshooting  these  xml  files  proved  very  time 
consuming. 

More  recently  the  team  has  been  investigating  HTMF5  which  has  some  promise,  but  as  yet  associated 
tool  chains  are  not  as  mature  as  with  Flash. 

3.7  Free  and  Open-Source  Software  (FOSS) 

Another  early  design  decision  was  to  use  as  much  free  and  open-source  software  (FOSS)  as  was  practical. 
While  the  actual  advantage  from  online  Flash  graphics  libraries  was  less  than  expected,  implementation 
of  the  overall  software  architecture  was  facilitated  greatly  by  FOSS. 

The  following  FOSS  software  have  been  used: 

1.  ‘‘MapServer  is  an  Open  Source  platform  for  publishing  spatial  data  and  interactive  mapping 
applications  to  the  web.  Originally  developed  in  the  mid- 1 990 ’s  at  the  University  of  Minnesota, 
MapServer  is  released  under  an  MIT-style  license,  and  runs  on  all  major  platforms  {Windows, 
Linux,  Mac  OS  X).  MapServer  is  not  a  full-featured  GIS  system,  nor  does  it  aspire  to  be.  [10]” 
VCEF  uses  Mapserver  to  generate  maps  for  tactical  overlays  etc.  in  its  in-house  emulations; 

2.  ’’BlazeDS  is  the  server-based  Java  remoting  and  web  messaging  technology  that  enables 
developers  to  easily  connect  to  back-end  distributed  data  and  push  data  in  real-time  to  Adobe® 
Flex®  and  Adobe  AIR™  applications  for  more  responsive  rich  Internet  application  (RIA) 


experiences.  [9]”  This  is  a  key  piece  in  linking  activated  Flash  interfaces  to  the  simulation  data 
network; 

3.  STAF/STAX  Software  Testing  Automation  Framework  and  STAF  Execution  Engine  are  used  in 
VCEL  to  automate  the  start-up/shut-down  processes  and  have  cut  those  times  by  close  to  50%. 

‘The  Software  Testing  Automation  Framework  (STAF)  is  an  open  source,  multi-platform,  multi¬ 
language  framework  designed  around  the  idea  of  reusable  components,  called  services  (such  as 
process  invocation,  resource  management,  logging,  and  monitoring).  STAF  removes  the  tedium 
of  building  an  automation  infrastructure,  thus  enabling  you  to  focus  on  building  your  automation 
solution.  The  STAF  framework  provides  the  foundation  upon  which  to  build  higher  level 
solutions,  and  provides  a  pluggable  approach  supported  across  a  large  variety  of  platforms  and 
languages.  [13]” 

“STAX  is  an  execution  engine  which  can  help  you  thoroughly  automate  the  distribution, 
execution,  and  results  analysis  of  your  test  cases.  STAX  builds  on  top  of  three  existing 
technologies,  STAF,  XML,  and  Python,  to  place  great  automation  power  in  the  hands  of 
testers.  [13]” 

4.  Apache  Active  MQ[8]  is  the  message  server  heart  of  the  software  design  which  links  the  wide 
variety  of  information  providers  and  consumers  together  (See  Figure  1). 

5.  Taksi  [16]is  a  video  capture/screen  capture  tool  for  recording  3D  graphics  applications  (such  as 
games). 

6.  OpenScales  is  an  open  source  (LGPL)  mapping  framework  written  in  ActionScript  3  and  Flex 
that  enables  developers  to  build  Rich  Internet  Mapping  Applications. 

7.  Apache  Tomcat  is  an  open  source  software  implementation  of  the  Java  Servlet  and  JavaServer 
Pages  technologies. 

3.0  CONCLUSIONS 

There  exists,  in  some  communities,  the  feeling  that  creating  an  operations  room  level  experimentation 
capability  is  a  very  expensive  proposition.  What  DRDC  has  shown  in  this  project  is  that  with  the 
judicious  use  of  commercial  and  FOSS  products  that  a  fully  activated  mock-up  of  a  naval  command  space 
can  be  developed  relatively  cheaply.  Total  research  funding  for  the  VCEL  project  was  under  a  million 
dollars  Canadian  with  an  in-house  team  of  about  four  technical  personnel  who  were  also  working  on  a 
variety  of  other  projects. 

The  main  cost  (money  and  time)  savers  that  enabled  the  project  were: 

1.  The  use  of  entertainment  industry  set  design  technologies  and  best  practices; 

2.  The  use  of  modified  commercial  gaming  technology  to  provide  baseline  capabilities  with  high 
usability; 

3.  The  use  of  COTS  and  consumer  grade  hardware; 

4.  The  use  of  FOSS  software  for  architecture  and  simulation  infrastructure;  and, 

5.  The  use  of  well-established  mature  interface  development  tools. 

None  of  the  above  technologies  were  a  perfect  match  to  our  problem  and  all  have  capability  gaps  that  had 
to  be  supplemented  with  in-house  development  and  solutions.  However,  the  base  of  available  simulation 


technology  (both  hardware  and  software)  is  substantial  and  growing.  By  focussing  custom  development 
on  the  gaps  it  is  possible  to  develop  cost-effective  experimentation  capabilities  for  complex  systems. 

The  final  caveat  is  that  maintenance  of  complex  systems  will  always  be  a  problem  and  picking  quality 
parts  and  quality  software  modules  is  likely  to  save  substantial  costs  in  the  long-term.  Note,  quality  does 
not  necessarily  mean  expensive. 
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■  What  is  the  problem?  [Going  to  sea  is  too  late  for  evaluation] 

■  The  Maritime  Capability  Evaluation  Laboratory  (MCEL)  project 
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What  is  the  problem? 


■  How  can  decision  makers  know  what  performance  effect  a  new  C2 
system  capability  proposal  will  have  before  incurring  acquisition  and 
engineering  integration  costs? 

■  How  do  you  de-risk  or  conduct  options  analysis  on  SOPs,  SORs,  or  new 
tech  prior  to  commitment...  are  you  sure  it's  worth  the  cost  before  you 
spend? 

■  Capability  development  is  mostly  done  at  the  sub-system  level, ...  and 
system  level  evaluation  occurs  at  project  close  when  the  money's  gone 
and  requirements  ore  set  in  concrete. 

■  Unable  to  understand  and  appreciate  those  capabilities  in  a  complete 
maritime  platform  context...  until  you  go  to  sea. 

You  can't  go  to  sea,  without  going  to  sea... 

...or  can  you? 
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Solution  Requirements 


■  Cheap 

■  Low  barrier  to  usage  so  most/all  projects  can  take  advantage 

■  Readily  available  to  operators  so  they  can  be  part  of  the  evaluation  process 

■  Reconfigurable/Adaptable 

■  Needs  to  be  able  to  support  range  of  concepts  (SOPS  to  Kit) 

■  Ability  to  measure  change  in  Warfighting  performance 

■  Need  to  have  measures  of  baseline  performance 

■  Human  performance  data  capture 
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Maritime  Capability  Evaluation  Lab  (MCEL) 


■  ADM(S&T)  Capital  Project 

■  Heading  for  Options  Analysis 

■  To  provide  an  enduring  shore-based  infrastructure  to 
support  capability  evaluation  for  naval  C2  systems. 

■  Two  re-configurable  stages  driven  by  a  common 
simulated  naval  combat  environment 

■  Integrated  HF  data  collection  and  analysis  facilities. 

■  Provide  the  capability  to  cheaply  test  the  impact  on 
mission  performance  of  changes  to  C2  systems  prior 
to  system  integration/implementation  costs. 

■  Networked  with  warfare  and  training  centres. 
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DRDC  Program  -  Victoria  Capability  Evaluation  Lab  (VCEL) 


■  Part  of  Victoria  Class  HSI  Optimization  study  -  originally  vVictoria 

■  Provide  an  experimental  facility  for  the  project  and  prototype 
technologies  for  C2  capability  evaluation. 

■  Realistic  Physical  constraints 

■  Interpersonal  modalities 

■  All  expected  sensor/information  stimuli 

■  Full  data  capture 

■  Exploration  of  low  cost  technologies 
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Technologies  Investigated 

■  Theatre/Television  set  design 

■  Simulation  Architecture 

■  COTS  gaming  technology 

■  COTS  data  collection  technology 

■  Rapid  interface  prototyping 
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Theatre/TV  Set  Design 


■  Use  of  theatre  style  staging  to  enable  under  floor  access 

■  50cm  off  floor 


■  Plywood  walls  and  ceilings 

■  Plywood,  mobile  consoles  for  flexibility 
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Software  System  Design 


Comms 


A/V 


Displays  or 
algorithms 
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Synthetic  Environment  Control 

■  Virtual  Machines  and  Thin  Clients 

■  Reduction  in  number  of  computers  to  maintain 

■  Single  place  for  configuration  control 

■  STAF/STAX  software  to  automate  startup/shutdown 
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COTS  Gaming  Technology 


■  Using  Dangerous  Waters  by  Sonalysts  as  main  simulation 

■  Multi-player  mode  emulating  major  operations  centre  consoles 

■  Provides  good  scenario  generation  and  control  interfaces 

■  Provides  good  scenario  elements  (maps,  units,  red  tactics  ...) 


■  Worked  with  Sonalysts  to  open  the  game  to  integration 
with  other  simulation  components  and  real  equipment. 


■  Evaluated  a  range  of  periscope  simulations 


DRDC  I  RDDC 


10 


COTS  Data  Collection 


■  SMi  Eye  tracking  glasses 

■  Low  light  surveillance  cameras 

■  MP3  audio  recording 


■  Kinect  tracking  of  participant  position  and  posture 
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Rapid  Prototyping 


■  Use  of  FLASH  to  mock-up 

■  Actual  console  screens  as  needed 

■  Prototype  user  interfaces  for  new  concepts 


■  Open  Source  software 

■  Map  Servers 

■  Message  handling  (ActiveMQ)  between 
applications 
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Conclusions 


■  Theatre  set  design  technologies  useful  -  cheap  and  can  be  done  in 
house. 

■  Gaming  Technology  can  cover  ~80%  of  needs 

■  Will  always  need  some  in-house  expertise  to  cover  the  integration  and  other  20% 

■  Flash  -  good  technology  but  took  more  in-house  development  of 
interface  objects  than  expected 

■  COTS  products 

■  Overall  they  are  good  quality  and  relatively  cheap 

■  But,  in-house  understanding  of  their  strengths  and  weaknesses  is  required  to  make 
effective  use  of  them.  There  are  always  gotchas! 
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